feat(spec): author DatasetSelectionSchema and parse the whole selection at the analytics dataset door - #19638

…rd package body stages, and stop the record under-reporting functions (objectstack-ai#19373)

Fixes objectstack-ai#17518

Clause-②: yes

Executes ruling **A′** — decision batch objectstack-ai#192 item 3, comment 5748934194,
maintainer 「192 同意」. Its two steps, its refusals (A and B) and its
fences are followed as written; every place where the tree made me read
the ruling rather than transcribe it is called out below.

Base of every reading in this body: regeneration commit `96dd3549ff6`,
the head of the SIXTH merge.

> ⚠️ **The readings below were brought to this head by the seat, not by
the round that first wrote them.** Two merge rounds have run since the
first draft. Each figure corrected here is named in the correcting
round's own report on card objectstack-ai#17518 — comment 5750725852 for the first,
5750987577 for the second — and the seat re-verified the head, the
regenerated index and mergeability itself before editing. Anything not
listed in those two reports is the original round's reading, unchanged.

## The confidence gap the ruling asked me to close first

「whether `effect` is required or defaulted on the declaration schema —
read it, ⛔ do not mint a value」

**Defaulted.** `FlowFunctionDeclarationSchema.effect` is
`FlowFunctionEffectSchema.default(DEFAULT_FLOW_FUNCTION_EFFECT)` where
that constant is `'pure'` (`automation/flow-function.zod.ts`). Measured,
not read off the source alone:
`FlowFunctionLoweredDeclarationSchema.safeParse({ handler: 'x' })`
succeeds and yields `{ handler: 'x', effect: 'pure' }`. The array member
of `functions` states `FlowFunctionEffectSchema.optional()` with **no**
default, so the two forms differ and neither is restated anywhere in
this diff — each JSON stage inherits its form's own optionality by
deriving from it.

That reading is what the producer writes: the bare-callable
normalisation uses `DEFAULT_FLOW_FUNCTION_EFFECT` and the array form
gets nothing.

## What landed

**`packages/spec/src/automation/flow-function.zod.ts`** —
`FlowFunctionLoweredDeclarationSchema` is exported (step 1), with its
`FlowFunctionLoweredDeclaration` / `…Parsed` aliases. It was a
module-local `const`, and `automation/index.ts`'s `export *` only
re-exports what is already exported.

**`packages/spec/src/stack.zod.ts`** — two new bodies **beside**
`AssembledPackageBodySchema`:

- `ArtifactStagePackageBodySchema` — the on-disk artifact stage.
`functions` entries are the lowered spellings, `hooks[].handler` is a
string.
- `RecordStagePackageBodySchema` — the registry record stage: literally
`ArtifactStagePackageBodySchema.extend({ functions: … })` with
`functions[].handler` optional in both the map-record form and the array
form, and nothing else.

`AssembledPackageBodySchema`, `composeStacks` and the `cannot drift`
invariant are ⛔ untouched: those callables are live on the stage the
assembled body declares itself for, and narrowing it would refuse a
published composition function's own output. Both new schemas carry the
same structural `z.ZodType` annotation as the assembled body, for the
two reasons recorded there (TS7056; a named alias turning `stack.zod`
into a shared chunk).

**`packages/spec/src/api/package-api.zod.ts`** — the installed-package
row's `manifest` is rebound to the record stage (step 1). The
`z.unknown()` override and the docblock defending it are gone, and the
sentence that ruling A step 5 assigns to this edit is corrected in
place: those two members are **not** why `ArtifactPackageSchema` and
`ObjectStackDefinitionSchema` publish no JSON Schema —
`src/stack.zod.ts` is not one of the subpath namespaces
`build-schemas.ts` walks, so neither is ever reached by the emit loop.

**`packages/objectql/src/registry.ts`** — step 2.
`withDeclaredFunctionEntries` rewrites a bare callable `functions` map
entry to `{ handler, effect: DEFAULT_FLOW_FUNCTION_EFFECT }` at the
assembly boundary, before `toRecordManifest` runs. `toRecordManifest`'s
structural rule is ⛔ untouched and no key is special-cased inside the
projection; the two spellings are simply made structurally equal ahead
of it. ⛔ No ref is minted, ⛔ no entry is dropped. The caller's manifest
is never mutated and a copy is made only when an entry really needed
rewriting.

## Two places where I read the ruling rather than transcribed it — both
stated so they can be overruled

1. **「`functions` entries the lowered declaration」 is implemented as
BOTH lowered members of `FlowFunctionEntrySchema`**, not only the record
one. `objectstack build` emits `{ myFn: 'myFn' }` for a bare entry and
`{ myFn: { handler: 'myFn', effect } }` for a declared one, so a stage
admitting only the record form would refuse artifacts this repo really
writes — the failure mode that withdrew letter B, one key across. Ruling
A′'s own step-4 control names both shapes (「a string and a lowered
record」). Measured: the artifact stage accepts a body carrying one of
each.
2. **The array member is transcribed, not derived.** `functions`' array
branch is declared inline inside the assembled body's own shape, and
narrowing it in place is the one thing this pair may not do. The
transcription's drift is guarded instead:
`stack-json-stage-package-body.test.ts` pins the authoring array entry's
key set equal to both JSON stages', so a key added there and not here
reddens by name.

## Acceptance, as ruling A′ lists it

| criterion | result |
|---|---|
| both bodies convert under `z.toJSONSchema` (self-test over the whole
body) | **YES** / **YES**; control: the assembled body still **NO**
(`Function types cannot be represented in JSON Schema`); probe controls
lit `z.string()` YES, dark `z.object({a: z.function()})` NO |
| the showcase-shaped manifest (`config.ts:244-249`) reports **2**
functions on the `GET /packages` row, the bare one as a handler-less
declaration | **2**:
`{"summarizeCompletedTask":{"effect":"pure"},"sweepProjectHealth":{"effect":"writes"}}`,
driven through the real `SchemaRegistry.installPackage` |
| `hooks` unchanged | unchanged: an inline handler is dropped (the key
is optional and admits that), a string handler survives verbatim. The
array `functions` form also keeps its entry:
`[{"name":"syncBilling","effect":"writes"}]` |
| `AssembledPackageBodySchema` / `composeStacks` / the invariant
untouched | untouched — no edit in those regions;
`assembled-package-body.test.ts` and
`compose-stacks-manifest-preserve.test.ts` stay green |
| the two `noted, not filed` corrections in the same edit | baseline
reason line: made TRUE by step 1 rather than reworded —
`automation/FlowFunctionLoweredDeclaration` is now in
`json-schema.manifest/automation.json`, so 「the lowered record …
publishes normally」 is now a fact. `package-api.zod.ts` docblock last
sentence: corrected in place, see above |

Stage separation, measured rather than asserted: the record stage
accepts the handler-less declaration and the **artifact** stage refuses
it; the assembled body accepts a live callable and **both** JSON stages
refuse it; both JSON stages still refuse an authoring glob and an
unknown key (`namesapce`). So the two keys moved from `unknown` to a
declaration, and nothing else moved.

## Reverse verification — two ablations, each restored with proof

Both ran against committed code, each with a `trap` restore, an on-disk
landing proof (anchor `grep -c` before/after plus a blob-hash change)
and a restore proof (`git hash-object` back to the HEAD blob, `git diff
HEAD` empty).

- **A1 — remove the producer normalisation**
(`toRecordManifest(withDeclaredFunctionEntries(manifest))` →
`toRecordManifest(manifest)`; anchor 1→0, injected 1, blob `b0af60d7…` →
`17b7c93c…`): `registry-package-manifest-serializable.test.ts` goes **1
failed / 15 passed**, naming the exact defect — `expected [
'sweepProjectHealth' ] to deeply equal [ 'summarizeCompletedTask', …(1)
]`. Restored blob `b0af60d7…`, diff empty.
- **A3 — collapse the record stage into the artifact stage**
(`jsonStageFunctionsKey(true)` → `(false)`; anchor 1→0, injected 2, blob
`60c13b43…` → `822bed8e…`): **2 failed / 79 passed** across two files —
`record accepts the handler-less declaration; ⛔ the ARTIFACT stage
refuses it` and `parses a row carrying the residual the projection
really produces`. So the one-key difference that IS the fourth stage is
load-bearing in both packages' pins. Restored blob `60c13b43…`, diff
empty.

No ablation is offered for 「both bodies convert」: that claim already
carries its discriminating control inside the same test file (the
assembled body must NOT convert), which is a lit/dark pair rather than
an assertion about itself.

## Tests and gates

All through `scripts/pm/os-verify-lock.sh` with
`OS_VERIFY_LOCK_SLOT=issue-17518`, verdicts read from the wrapper's own
`VERDICT command-exit` line and never a bare `$?`; every exit code
captured before any pipe. Wall-clock figures in the logs are SHARED-BOX
seconds.

- `pnpm --filter @objectstack/spec test` — **513 files / 14971 tests
passed, 1 todo** — the FULL suite, re-run on this head because the sixth
merge carried 128 commits of base movement including breaking spec
changes
- `pnpm --filter @objectstack/objectql test` — **303 files / 5057 tests
passed**
- `pnpm --filter @objectstack/runtime exec vitest run --maxWorkers=2`
over the package-door / artifact population, enumerated by a name match
on `packages/runtime` for `package` or `artifact` so the population is
reproducible — **39 files / 512 tests passed**. ⚠️ The first attempt
exited 1 in 2 seconds and is recorded as NOT a red: the paths were
repo-root-relative while `pnpm exec` runs at the package root, and the
repo's own guard said so in words (`FILTER SELECTED NOTHING — 39 of the
39 path(s) you named will run no tests`). Re-run with package-relative
paths for the reading above.
- `pnpm --filter @objectstack/spec --filter @objectstack/objectql
typecheck` — exit 0; both test layers compile (spec **53 files / 257
errors / 142 pins**; objectql **40 / 234 / 65**, unchanged). ⚠️ The spec
ledger moved from 54 / 259 / 144 by main's objectstack-ai#19364 arriving in a merge, ⛔
not by this PR.
- `pnpm --filter @objectstack/spec --filter @objectstack/objectql
typecheck` — both exit 0 on this head; the debt ledgers held shrink-only
(spec 53 files / 257 errors / 142 pinned signatures; objectql 40 / 234 /
65).
- `pnpm --filter @objectstack/spec build` exit 0 (34/34 declared `.d.ts`
present, `check-dts-references` resolved 378/378), and the whole
`@objectstack/runtime` dependency closure was rebuilt first, so nothing
below read a dist stale against 128 commits of main.

**Gates.** `node scripts/pm/dispatch-gates.mjs --commands --repo
objectstack-ai/objectstack` derived from this tree, every command run
with its exit code written to a file, reconciled with `--ran`: **116
derived, 114 run, 2 NOT-MEASURED, 0 UNRUN**, and the tool's own verdict
line says so. **113 exit 0.** The two NOT-MEASURED are the tool's
DERIVED classification of an exit 3; a third measured nothing too, and
the tool cannot see it because its refusal code is 2. ⛔ None of the
three is a finding:

- `check:dual-build-cjs-loads` — exit **3**, its own `PREREQUISITE NOT
MET … ⛔ This is NOT a pass: nothing was measured` (66 packages have no
`dist`; it wants a whole-repo build).
- `check:type-check-debt` — exit **3**, same shape, same wording, wants
the full package closure built.
- `check-engine-split-ratio --days 90` — exit **2**, refuses on a
shallow clone whose oldest visible commit sits inside the 90-day window.
It says a ratio derived there would be 「real, plausible and WRONG」.

A fourth, `check:skill-examples`, first exited 1 on an unbuilt
`packages/client-react`; after building that package it re-runs
**green** — 258 prose examples type-check across 3 surfaces. Both
readings are stated here, and the reconciliation record carries ONE of
them — the green re-run — because the tool flags a doubly-recorded
family and says to make the record state one thing. The re-derivation on
the final head yields **116** families: `check:api-surface-declarations`
is gone (retired upstream by objectstack-ai#19024 mid-round) and
`check:gitlink-declared` is new, run green. No family is left unrun.

Ratchet families re-run after the last merge, on `96dd3549ff6`:
`check:generated` (all 15 artifacts up to date), `check:api-surface`,
`check:authorable-surface`, `check:export-origins`,
`check:declaration-map`, `check:docs`, `check:skill-refs`,
`check:entry-nameability`, `check:dual-source-exports`,
`check:spec-changes`, `check:spec-parsed-alias`,
`check:published-files`, `check:nul-bytes`,
`check:cross-package-test-inputs`, `check:test-source-alias`,
`check:type-check-coverage` — all exit 0. Control characters: `grep
-naP` over every file I hand-edited returns nothing (exit 1).

## Generated artefacts in this diff, and why each moved

- `json-schema.manifest/automation.json`,
`authorable-surface/automation.json`,
`authorable-defaults/automation.json`, `api-surface/*`,
`export-origins/*`, `declaration-map/automation.json`,
`content/docs/references/**` — the new exports, regenerated by the
package's own `gen:` scripts. `authorable-defaults` records
`automation/FlowFunctionLoweredDeclaration:effect = "pure"`, which is
the confidence-gap reading in ledger form.
- `packages/spec/dropped-refinements.baseline.json` — four `api/*`
entries each gain one site (`…manifest.hooks.element.object`), counts
569 → 573. Cause: the record stage **declares** `hooks` where
`z.unknown()` declared nothing, so `HookSchema`'s `object` refinement
now reaches the runtime and not the published file. The ledger is
hand-edited by design and the build printed the exact delta.
- `skills/objectstack-platform/references/_index.md` — one generated
line listing `stack.zod.ts`'s exports.

## `skills/**` readings, and the landing tier

This diff touches `skills/objectstack-platform/references/_index.md`, so
the PR is **governed, Tier H** on its file list. ⛔ It stays a draft and
no AI seat merges, queues or arms auto-merge on it.

Both readings the skills rule requires, at merge base `c334ba0f3a6`:

- **changed file, whole file**: 41 lines before, 41 after — net **0**.
The diff is one regenerated line.
- **package total (sum of every `SKILL.md`)**: 6145 before, 6145 after —
net **0**.

`node scripts/check-skills-token-ratchet.mjs` exits 0 and classifies
this file as **generator-owned (measured, not ratcheted)**, so no
authored ceiling is charged.

## Clause ②, and the changeset is not one package's

`Clause-②: yes`, and two changesets because two published packages move:

- `@objectstack/spec` — **minor**. New exports, and the two
installed-package responses move from `z.unknown()` on `functions` /
`hooks` to declared JSON shapes. That is a narrowing on a published
declaration; what it does NOT withdraw is measured, on real producers:
the showcase shape, the array form and the already-lowered body an
artifact boot installs all parse.
- `@objectstack/objectql` — **patch**. `GET /packages` reports functions
it previously dropped. No API is added or removed; a read door stops
under-reporting. Grade it up if a payload gaining entries reads as minor
to the reviewer.

## Serial and merge state, re-taken by this seat

Changed-file map re-taken first-hand over all **33** open PRs (271 file
rows) rather than inherited. LIT control
`packages/spec/src/ui/action-params.zod.ts` resolves to objectstack-ai#19315; DARK
control `packages/spec/src/zzz-no-such.zod.ts` resolves to nothing.

- `packages/spec/src/automation/flow-function.zod.ts`,
`packages/spec/src/api/package-api.zod.ts`,
`packages/objectql/src/registry.ts` — **free**.
- `packages/spec/src/stack.zod.ts` — held by objectstack-ai#18482, objectstack-ai#19147, objectstack-ai#19314, all
below A′'s region. objectstack-ai#19147 landed during this round and merged cleanly
here (its `stack.zod.ts` hunk is a comment).
- `packages/spec/dropped-refinements.baseline.json` — also written by
objectstack-ai#19147 (landed, resolved here) and by the still-open **objectstack-ai#19335**, which
rewrites the same `measured` header and adds entries. That is a
line-level contention on a ledger whose correct value is recomputable:
whoever lands second re-runs `pnpm --filter @objectstack/spec build` and
re-applies the delta it prints. ⛔ Not a semantic collision.

`origin/main` has been merged **six** times on this branch. `objectstack-ai#19024`
(which retired `api-surface-declarations/`) came in early, which is why
no `api-surface-declarations/*.txt` appears in this diff. The fifth
merge brought **objectstack-ai#19363**, a BREAKING spec change. The **sixth** merge,
the head of this body, brought **128 commits** — so the full spec suite
was re-run rather than only the generated gates.

⛔ `scripts/pm/os-regen-merge.sh` was NOT used in either round — its
`rerun` arm is re-entrant and commits a revert of the operator's own
regeneration, filed as **objectstack-ai#19392**. Steps 1–3 of its documented order
were performed by hand, against a merge base captured BEFORE the merge
and an `origin/main` fetched into an OWNED ref so a sibling's fetch
could not move the target mid-round.

**The sixth merge decided THREE paths, and only one of them was a
conflict.** That gap is worth stating, because resolving only what a
conflict probe names would have landed a silent loss:

| path | routed | what the merge did | how it was resolved |
|:--|:--|:--|:--|
| `content/docs/references/index.mdx` | `merge=os-regen` | driver
deferred it, exit 0 — **main's side silently dropped** (merged blob
`6290447bd9a` == ours, != theirs `7e1f9b6f13e`) | main's side restored
into the WORKING TREE ONLY, then regenerated whole |
| `content/docs/references/api/package-api.mdx` | `merge=os-regen` |
same — **main's side silently dropped** (merged `988bedaa480` == ours,
!= theirs `d09cd420711`) | same |
| `packages/spec/dropped-refinements.baseline.json` | **not** routed |
exit 1 — the only real text conflict, one hunk, confined to three
summary counters in the `measured` header | both sides' entries unioned,
then the build adjudicated |

⚠️ **`package-api.mdx` appears in NO conflict list and never could.** It
text-merges cleanly driver-free, so a GitHub-condition probe cannot name
it; only the both-edited ROUTED set, computed per file against the
pre-merge base, finds it — which is exactly what `os-regen-merge.sh`
step 2 specifies and what the driver's own `$GIT_DIR/os-regen-pending`
record listed.

**The regenerated docs are the UNION, proven in both directions**
(added/removed line multisets compared as sets): `package-api.mdx`
identical at 20 and 14 lines; `index.mdx` identical at 12 and 6 lines,
excluding the two running-total lines — a union MUST move a total
neither side moves alone, so their disagreement is the signature of a
correct union rather than a failure, and the line counts already matched
(16/16, 10/10) before excluding them. The total is **re-derived, not
arithmetic**: base 1533, this branch alone 1534, main alone 1534, merged
tree **1535**, and 1535 is what `gen:schema` itself reports for the
merged sources. Main brought `DatasetSelection`, `DatasetCompareTo` and
`DatasetTotals` and retired `KernelSecurityScanResult` /
`KernelSecurityVulnerability`; this branch brought
`FlowFunctionLoweredDeclaration`. All survive, asserted through the
published export map of the freshly built dist with a dark control (an
invented export name reads undefined).

**The ledger was resolved by hand, and that is the only route
available.** `dropped-refinements.baseline.json` is hand-edited BY
DESIGN with no `gen:` script — its own description states why: *"a
generator would let a new gap be admitted by running a command instead
of by a decision, which is the silence this ledger exists to end."* The
build VALIDATES it bidirectionally and refuses; it never writes it. Both
sides' entries were unioned (union keys missing from the merged file:
**none**; merged keys not in the union: **none**; `api/DatasetSelection`
arrived from main via objectstack-ai#19638 and survives; main's removal of the
`fields.out.keyType` sites is kept — **nine** site lines at the merge
base, zero at this head and zero on main (lit control: 204 `"sites"`
keys at base; dark control 0). ⚠️ The merge round's own prose said
*five*; that was a narrative miscount caught by the merge-delta review
and re-counted by the seat. The FILE was always right), then
`gen:schema` adjudicated and measured 565 dropped sites across 205
published schemas — the union as resolved. One counter the build
corrected: `refinementSitesThatDidProject` read 357 and the build
measures 366.

⚠️ **That correction is filed as objectstack-ai#19681**, because nothing in the
repository would have caught it: two of the four `measured` counters
have no reader anywhere (lit control — the other two have two readers
each, dark control 0), so they can hold any number and every gate stays
green.

## Acceptance notes

- **noted, not filed**: regenerating
`packages/spec/api-surface-declarations/ui.txt` produced a 184-line
change that is a pure permutation of its own content — the same union
members in a different order, `0 removed, 0 added, 35 reshaped`.
Verified as a precedented shape rather than a defect: commit
`24d622b94b8`, a spec change touching **zero** files under
`packages/spec/src/ui/`, moved the same file by 5 lines whose sorted
content is byte-identical. The whole artefact was retired upstream by
objectstack-ai#19024 mid-round, so nothing of it survives in this diff and the
population is gone. **Carrier: none — the file no longer exists.**
- **noted, not filed**: `packages/objectql`'s tests resolve
`@objectstack/metadata-protocol` from `dist`, so after merging upstream
objectstack-ai#19277 the seven assertions in
`protocol-install-package-enable-on-install.test.ts` failed against a
stale build of a package this PR never touches; building that one
package turns all seven green. A local-environment reading, not a repo
defect, and `check:test-source-alias` already owns the aliased/unaliased
ledger this sits in. **Carrier: the next seat that runs objectql's suite
after a merge — it will see the same red and should build the dependency
before reading it as a finding.**

## 维护者速读(草稿)

**改了什么** —— 一个包的「包体」在平台里其实要经过四个阶段:作者写的、内存里装配好的、落盘成 artifact
的、注册表记录下来的。前两个早有声明,后两个从来没有。这次把后两个补上:`ArtifactStagePackageBodySchema`(落盘
artifact)和
`RecordStagePackageBodySchema`(注册表记录),放在既有的装配体**旁边**,装配体一个字不动。同时修好一个生产者缺陷:`GET
/packages` 以前会把「裸写的函数」整条漏报,现在两种写法都报。

**为什么改** —— 两件事各有代价。其一,装配体里有两个键(`functions`、`hooks`)声明了「可以是一个活的函数」,而
JSON Schema 表达不了函数,于是**任何嵌入它的接口都会整份丢掉自己的 JSON Schema**;读 API
只能把这两个键写成「什么都收、不检查」。其二,我们自己发布的 showcase 声明了 2 个函数,而 `GET /packages` 只报 1
个——机器可读的读门把事实说少了。

**风险与代价(含回滚)** ——
风险集中在一处:那两个键从「什么都收」变成「按声明收」,理论上可能拒掉今天能读的行。已实测三种真实生产者(showcase
的写法、数组写法、artifact 启动装回来的写法)全部照常通过,并且用两次消融证明了这些断言真的会红而不是摆设。⛔ 装配体与
`composeStacks` 未动,所以 `os dev` / `os serve` 的行为不受影响——这正是上一版裁决 B
被撤回的原因,这次没有重蹈。回滚:两个 spec 改动与 objectql 改动互相独立,`git revert`
任一半都不会让另一半变红;最小回滚是把 `package-api.zod.ts` 的那一行绑回装配体,新声明留着不用。

**席位意见** ——

**你要做的** —— 这个 PR 的文件里有一份 `skills/**` 的生成文件,按规则整单属于 Tier
H,**只有你(或你授权的批准)能让它落地**;AI 席位不会合并、不会排队、不会解除 draft。请看两点:①
`@objectstack/objectql` 我打的是 `patch`,理由是「读门修复、不增删 API」,若你认为「载荷多出条目」应算
minor,说一声即可改;② `functions` 的声明式阶段我按「两种 lowered 写法都收」实现(理由写在上面第 1
条),如果裁决本意是只收记录式那一种,也请直接说,那会让 `objectstack build` 今天写出的一种 artifact 被拒。

---
_Generated by [Claude
Code](https://claude.ai/code/session_01LvwGppdonww4zGLWZo5rho)_

---
_Generated by [Claude Code](https://claude.ai/code)_

---------

Co-authored-by: Claude <noreply@anthropic.com>